In the emerging Software as a Service (“SaaS”) market, the majority of the processing occurs at the server and the data is centralized to take advantage of the anywhere, anytime connectivity that the Internet provides. The power of the connectivity to and sharing of information is leading to massively scalable applications that support hundreds of thousands, up to hundreds of millions of users.
Massively scalable applications are creating many new challenges in managing the users and data in an automated fashion. The ability to manage user load is particularly critical in data-heavy applications, such as email, file storage, and online backup. A user load is a load on a given server or servers using one or more services and an overall load includes the utilization of processor, memory, I/O reads, writes and transactions per second, network, disk space, power, and application or applications. To manage overall loads, currently administrators are forced to manually move these data heavy user loads between servers which is time consuming and inefficient. Additionally, these manual moves of user loads can lead to interruption in service to users.
Load balancers have been developed, but they reside at the edge of the network in front of application servers and are only available to take and split incoming traffic data between servers based on low level metrics. These low level metrics consist of factors, such as the number of network connections on a server or how fast a server is responding to HTRP requests, and are unrelated to either user loads or overall loads on the servers. These prior load balancers work well for web servers where all of the data on each server is identical or “stateless,” but do not work in applications where each user has unique data, such as in the emerging SaaS market or Web 2.0, where the data on each server is unique to each user or “stateful” and thus is dynamically changing for each user. Additionally, these load balancers do not work well were each of the user loads utilize different applications.